[apm] Update APM feature roles docs #4193

colleenmcginnis · 2024-08-23T13:52:36Z

In #3980 (comment), @endorama suggested that we clarify the docs on APM feature roles. Let me know what you think of this approach. I attempted to clarify how feature roles work and which privileges are required for different use cases by:

Reframing the intro to feature roles to emphasize that the what we're recommending is to create several feature-related roles and then assign one or more of these roles to each user (or group) based on which features the user needs to access.
Providing an example of when a user might need to be assigned multiple roles from this list of feature roles (this needs work).
Emphasizing that the Central configuration management role is required in most cases (unless central configuration management is explicitly disabled by the org/user).
Putting all roles on a single page to increase the likelihood that the reader will see the context provided in the intro paragraphs.
Adding a step at the end of the instructions for creating each role to remind users that they can assign other APM feature roles to the users as needed.

Here's what hasn't been addressed yet:

Verify that the privileges listed in each role are correct and reflect the latest version of APM / Elastic as a whole. (I'm not sure how to go about confirming or testing these.)
Are these the right feature roles? Are any missing? Should any be removed?

Here's the PR preview: https://observability-docs_bk_4193.docs-preview.app.elstc.co/guide/en/observability/master/apm-feature-roles.html

@endorama let me know what you think of this approach. If we're on the right track, we can iterate in this PR.

cc @simitt @carsonip

github-actions · 2024-08-23T13:52:49Z

A documentation preview will be available soon.

🔨 Buildkite builds
📚 HTML diff
📙 Preview page

Request a new doc build by commenting

Rebuild this PR: run docs-build
Rebuild this PR and all Elastic docs: run docs-build rebuild

_{run docs-build is much faster than run docs-build rebuild. A rebuild should only be needed in rare situations.}

_{If your PR continues to fail for an unknown reason, the doc build pipeline may be broken. Elastic employees can check the pipeline status here.}

carsonip

Verify that the privileges listed in each role are correct and reflect the latest version of APM / Elastic as a whole. (I'm not sure how to go about confirming or testing these.)
Are these the right feature roles? Are any missing? Should any be removed?

Our team should confirm the required privileges. IIUC points to update:

missing privilege to query cluster uuid
mark api key role as deprecated

docs/en/observability/apm/feature-roles.asciidoc

endorama

Left some comment but overall is already a great improvement 🤩

docs/en/observability/apm/feature-roles.asciidoc

endorama · 2024-09-06T16:18:59Z

I'll take on me to:

Verify that the privileges listed in each role are correct and reflect the latest version of APM / Elastic as a whole. (I'm not sure how to go about confirming or testing these.)
Are these the right feature roles? Are any missing? Should any be removed?
confirm the required privileges to query cluster uuid.
confirm if API key role should be mark as deprecated (and if yes when deprecation happened)

endorama · 2024-09-20T08:05:10Z

TL;DR: querying cluster uuid requires monitor permission if Elasticsearch security features are enabled or none if not.

Today I researched how to query cluster UUID. My understanding is that apm-server queries Elasticsearch on the / path to obtain it. From the documentation

f the Elasticsearch security features are enabled, you must have the monitor, manage, or all cluster privilege to use this API.

endorama · 2024-09-20T08:25:07Z

Regarding the Api Key role, my understanding is that is only needed for the apikey command in apm-server. This command has never been labelled GA, is deprecated since 8.6.0 and will be removed. For this command to work priviliges manage_own_api_key are required. The users should leverage Kibana and Elasticsearch APIs instead.

endorama · 2024-09-23T15:36:35Z

I've completed checking for roles/permissions.

🟩 source map role correctly covers using APM source maps in RUM
🟩 privileges checks are allowed without authentication in APM use case (checking own permissions)
🟨 cluster UUID checks are always performed as part of APM server startup preconditions; this check may require monitor permissions if Elasticsearch Security is enabled
🟨 if Tail Based Sampling is enabled a license check is performed; this check requires monitor permissions
🟩 writer role correctly covers ingesting data from APM Server to Elasticsearch
🟥 central config role should correctly cover using Central Config, but I was not able to test it. The current docs mention setting allow_restricted_indices to true but I was not able to understand how to do that from the Roles UI when creating a role. @colleenmcginnis if you have any clarification/input here I'd like to test this one too.

Details

SEARCH .apm-source-map here: covered by source map role
- used here
HasPrivileges request here: does not require privileges
- HasPrivileges API
  - All users can use this API, but only to determine their own privileges.
    To check the privileges of other users, you must use the run as feature.
GET / here:
- check is performed as part of every run when checking preconditions here
GET _license here
- only requested if tail sampling is enabled here
- Get license APIrequires monitor permissions
Bulk here (in go-docappender):
- Bulk is performed as part of a docappender
  - BulkIndexerItems are added to the configured appedender here
SEARCH .apm-agent-configuration here: covered by central config role but I wasn't able to test it because in the UI there seems to be no way to set allow_restricted_indices=true

colleenmcginnis · 2024-09-23T20:04:29Z

The current docs mention setting allow_restricted_indices to true but I was not able to understand how to do that from the Roles UI when creating a role. @colleenmcginnis if you have any clarification/input here I'd like to test this one too.

It looks like that setting was added about 7 months ago by @carsonip in 4834fce. Maybe Carson can provide context? (The related PR and issue don't provide much information.)

colleenmcginnis · 2024-09-24T00:37:45Z

Also @endorama ...

cluster UUID checks are always performed as part of APM server startup preconditions; this check may require monitor permissions if Elasticsearch Security is enabled

... can you confirm if this just for the writer role or if it is necessary for all roles? The phrase APM server startup preconditions makes me think it's necessary to do anything, but I wanted to make sure. Also is monitor a cluster or index privileges? It looks like there are both in https://elastic.co/guide/en/elasticsearch/reference/master/security-privileges.html.

endorama · 2024-09-25T13:13:42Z

@colleenmcginnis Sorry for the confusion! monitor refers to the cluster privilege. Being it a cluster privilege it's not necessary on all roles but at least 1 needs to have it if:

security is enabled
tail based sampling is used

I think adding it to the writer role make sense as the checks are performed as part of the usual APM server startup process.

carsonip · 2024-09-25T16:27:18Z

The current docs mention setting allow_restricted_indices to true but I was not able to understand how to do that from the Roles UI when creating a role

Apparently there is no option to set allow_restricted_indices in roles UI. Roles created in this UI will have allow_restricted_indices default to false. e.g.

{
  "agentcfg_1": {
    "cluster": [],
    "indices": [
      {
        "names": [
          ".apm-agent-configuration"
        ],
        "privileges": [
          "read"
        ],
        "field_security": {
          "grant": [
            "*"
          ],
          "except": []
        },
        "allow_restricted_indices": false
      }
    ],
    "applications": [],
    "run_as": [],
    "metadata": {},
    "transient_metadata": {
      "enabled": true
    }
  }
}

However, if you create the API key from API Key UI, under "Control security privileges", the example config in the textbox will have allow_restricted_indices configurable. The last time I tested this, agentcfg requires allow_restricted_indices to be true. @endorama would be good if you confirm that again.

endorama · 2024-09-26T07:27:09Z

@carsonip I can confirm it's broken with allow_restricted_indices: false. I'll test it with the API key as described.

endorama · 2024-09-27T09:34:29Z

@carsonip Tested and it's working! While testing I also went down a rabbit hole for the API Key format that I documented in #4312 so we can address it separately.

Co-authored-by: Edoardo Tenani <[email protected]>

endorama

🤩 looks great! Thank you!

mdbirnstiehl

Some little things and some questions about heading hierarchy.

docs/en/observability/apm/security/elastic-stack/feature-roles.asciidoc

Co-authored-by: Mike Birnstiehl <[email protected]>

colleenmcginnis · 2024-10-07T20:13:35Z

Some little things and some questions about heading hierarchy.

Thank you! I messed up the headings when I caught up to main today. 🙈

mdbirnstiehl

🦭 👍

* initial attempt * restructure * fix build * address initial feedback * deprecate api key role * fix typo * add use cases * reframe what we mean by users * fix redirect * add monitor privilege * clean up structure * Update docs/en/observability/apm/feature-roles.asciidoc Co-authored-by: Edoardo Tenani <[email protected]> * update docs/en/observability/apm/feature-roles.asciidoc * Update feature-roles.asciidoc * update docs/en/observability/apm/feature-roles.asciidoc * use roles api in central config section * apply suggestions from code review Co-authored-by: Mike Birnstiehl <[email protected]> --------- Co-authored-by: Edoardo Tenani <[email protected]> Co-authored-by: Mike Birnstiehl <[email protected]> (cherry picked from commit d6a61fd) # Conflicts: # docs/en/observability/apm/feature-roles.asciidoc

* initial attempt * restructure * fix build * address initial feedback * deprecate api key role * fix typo * add use cases * reframe what we mean by users * fix redirect * add monitor privilege * clean up structure * Update docs/en/observability/apm/feature-roles.asciidoc Co-authored-by: Edoardo Tenani <[email protected]> * update docs/en/observability/apm/feature-roles.asciidoc * Update feature-roles.asciidoc * update docs/en/observability/apm/feature-roles.asciidoc * use roles api in central config section * apply suggestions from code review Co-authored-by: Mike Birnstiehl <[email protected]> --------- Co-authored-by: Edoardo Tenani <[email protected]> Co-authored-by: Mike Birnstiehl <[email protected]> (cherry picked from commit d6a61fd)

* initial attempt * restructure * fix build * address initial feedback * deprecate api key role * fix typo * add use cases * reframe what we mean by users * fix redirect * add monitor privilege * clean up structure * Update docs/en/observability/apm/feature-roles.asciidoc Co-authored-by: Edoardo Tenani <[email protected]> * update docs/en/observability/apm/feature-roles.asciidoc * Update feature-roles.asciidoc * update docs/en/observability/apm/feature-roles.asciidoc * use roles api in central config section * apply suggestions from code review Co-authored-by: Mike Birnstiehl <[email protected]> --------- Co-authored-by: Edoardo Tenani <[email protected]> Co-authored-by: Mike Birnstiehl <[email protected]> (cherry picked from commit d6a61fd) Co-authored-by: Colleen McGinnis <[email protected]>

* [apm] Update APM feature roles docs (#4193) * initial attempt * restructure * fix build * address initial feedback * deprecate api key role * fix typo * add use cases * reframe what we mean by users * fix redirect * add monitor privilege * clean up structure * Update docs/en/observability/apm/feature-roles.asciidoc Co-authored-by: Edoardo Tenani <[email protected]> * update docs/en/observability/apm/feature-roles.asciidoc * Update feature-roles.asciidoc * update docs/en/observability/apm/feature-roles.asciidoc * use roles api in central config section * apply suggestions from code review Co-authored-by: Mike Birnstiehl <[email protected]> --------- Co-authored-by: Edoardo Tenani <[email protected]> Co-authored-by: Mike Birnstiehl <[email protected]> (cherry picked from commit d6a61fd) # Conflicts: # docs/en/observability/apm/feature-roles.asciidoc * fix conflicts --------- Co-authored-by: Colleen McGinnis <[email protected]>

colleenmcginnis added 2 commits August 22, 2024 16:59

initial attempt

b5818d5

restructure

c2eb01b

colleenmcginnis added backport-8.14 Automated backport with mergify backport-8.15 Automated backport with mergify labels Aug 23, 2024

colleenmcginnis requested a review from endorama August 23, 2024 13:52

colleenmcginnis self-assigned this Aug 23, 2024

carsonip self-requested a review August 23, 2024 14:04

fix build

ed83d71

carsonip reviewed Aug 28, 2024

View reviewed changes

address initial feedback

671382f

endorama reviewed Sep 6, 2024

View reviewed changes

docs/en/observability/apm/feature-roles.asciidoc Outdated Show resolved Hide resolved

docs/en/observability/apm/feature-roles.asciidoc Outdated Show resolved Hide resolved

docs/en/observability/apm/feature-roles.asciidoc Outdated Show resolved Hide resolved

bmorelli25 added the backport-8.x Automated backport to the 8.x branch with mergify label Sep 12, 2024

colleenmcginnis added 5 commits September 20, 2024 12:43

Merge branch 'main' into issue-3980

ae55f4e

deprecate api key role

90e213e

fix typo

8cdb6cb

add use cases

571a58c

reframe what we mean by users

c8fb50d

bmorelli25 added needs-writer-review needs-dev-review labels Sep 20, 2024

fix redirect

e0be572

add monitor privilege

0e65820

colleenmcginnis requested a review from a team as a code owner September 25, 2024 19:23

endorama mentioned this pull request Sep 27, 2024

Fix apm agentcfg api key privileges #4083

Merged

colleenmcginnis requested a review from endorama September 27, 2024 17:16

colleenmcginnis and others added 5 commits October 2, 2024 16:01

Update docs/en/observability/apm/feature-roles.asciidoc

21ed200

Co-authored-by: Edoardo Tenani <[email protected]>

update docs/en/observability/apm/feature-roles.asciidoc

10ba5c9

Update feature-roles.asciidoc

03fbafd

update docs/en/observability/apm/feature-roles.asciidoc

9ba0f02

use roles api in central config section

d3f19ba

endorama previously approved these changes Oct 4, 2024

View reviewed changes

colleenmcginnis removed the needs-dev-review label Oct 4, 2024

colleenmcginnis linked an issue Oct 4, 2024 that may be closed by this pull request

[Request]: apm_writer user does not have enough permissions #3980

Closed

Merge branch 'main' into issue-3980

440d2ce

colleenmcginnis dismissed endorama’s stale review via 440d2ce October 7, 2024 13:22

mdbirnstiehl reviewed Oct 7, 2024

View reviewed changes

apply suggestions from code review

72c3693

Co-authored-by: Mike Birnstiehl <[email protected]>

mdbirnstiehl approved these changes Oct 7, 2024

View reviewed changes

colleenmcginnis merged commit d6a61fd into elastic:main Oct 7, 2024
3 checks passed

colleenmcginnis deleted the issue-3980 branch October 7, 2024 21:09

mergify bot mentioned this pull request Oct 7, 2024

[8.14](backport #4193) [apm] Update APM feature roles docs #4355

Merged

2 tasks

mergify bot mentioned this pull request Oct 7, 2024

[8.15](backport #4193) [apm] Update APM feature roles docs #4356

Merged

2 tasks

mergify bot mentioned this pull request Oct 7, 2024

[8.x](backport #4193) [apm] Update APM feature roles docs #4357

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[apm] Update APM feature roles docs #4193

[apm] Update APM feature roles docs #4193

colleenmcginnis commented Aug 23, 2024 •

edited

Loading

github-actions bot commented Aug 23, 2024

carsonip left a comment

endorama left a comment

endorama commented Sep 6, 2024 •

edited

Loading

endorama commented Sep 20, 2024

endorama commented Sep 20, 2024

endorama commented Sep 23, 2024 •

edited

Loading

colleenmcginnis commented Sep 23, 2024

colleenmcginnis commented Sep 24, 2024 •

edited

Loading

endorama commented Sep 25, 2024

carsonip commented Sep 25, 2024

endorama commented Sep 26, 2024

endorama commented Sep 27, 2024

endorama left a comment

mdbirnstiehl left a comment

colleenmcginnis commented Oct 7, 2024

mdbirnstiehl left a comment

[apm] Update APM feature roles docs #4193

[apm] Update APM feature roles docs #4193

Conversation

colleenmcginnis commented Aug 23, 2024 • edited Loading

github-actions bot commented Aug 23, 2024

carsonip left a comment

Choose a reason for hiding this comment

endorama left a comment

Choose a reason for hiding this comment

endorama commented Sep 6, 2024 • edited Loading

endorama commented Sep 20, 2024

endorama commented Sep 20, 2024

endorama commented Sep 23, 2024 • edited Loading

colleenmcginnis commented Sep 23, 2024

colleenmcginnis commented Sep 24, 2024 • edited Loading

endorama commented Sep 25, 2024

carsonip commented Sep 25, 2024

endorama commented Sep 26, 2024

endorama commented Sep 27, 2024

endorama left a comment

Choose a reason for hiding this comment

mdbirnstiehl left a comment

Choose a reason for hiding this comment

colleenmcginnis commented Oct 7, 2024

mdbirnstiehl left a comment

Choose a reason for hiding this comment

colleenmcginnis commented Aug 23, 2024 •

edited

Loading

endorama commented Sep 6, 2024 •

edited

Loading

endorama commented Sep 23, 2024 •

edited

Loading

colleenmcginnis commented Sep 24, 2024 •

edited

Loading